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ABSTRACT 

This paper presents results from using an asynchronous Web-based discussion forum coupled with 
an integrated student peer rating system as one component of an introductory Information Systems 
(IS) course with high enrollment (e.g. a class with over 100 students). There are two major issues 
with the typical introductory IS course: it covers too many topics , and it is difficult to engage students 
in active learning in a classroom environment due to large class sizes. To address these issues, an 
open source discussion forum was modified and usedfor topical discussion outside of the classroom. 
Students were allowed to participate in a discussion about topics of their choice and were encouraged 
to rate their peers' contribution in forum. Overall , students evaluated the online forum discussion 
positively and provided suggestionsfor improvement. 


INTRODUCTION 

Using information systems (IS) is a pervasive 
organization phenomenon; it is hard to imagine 
that any organization could survive in today’s 
business environment without support from 
some type of information system. A basic under¬ 
standing of IS, therefore, is usually a requirement 
for undergraduate business students. In a typi¬ 
cal undergraduate business program, students 
are first exposed to IS in an introductory course. 
As Bakke and his colleagues (Bakke, et al, 2007) 
point out, this introductory course is often too 
large, covers “a vast array of subjects”, focuses 
“too broadly on most topics”, enrolls students 
with “a variety of backgrounds and interests”, 
and is often perceived as “impersonal”. Moreover, 
students “often enroll in these courses primarily 
to satisfy graduation requirements rather than 
to satisfy an inherent interest in the subject mat¬ 
ter. Hence, neither students nor professors are 
satisfied with the learning experience and, more 
importantly, graduates from business schools do 
not master fundamental IS skills before entering 
the professional world.” 


In this paper, we illustrate how an online fo¬ 
rum discussion can be used to improve students’ 
learning process and learning outcomes in such 
a large introductory course. An online discus¬ 
sion forum (or simply a forum) is a Web-based 
application that provides the ability to host a 
persistent threaded discussion. Three objectives 
provide guidance for designing an online forum 
discussion: 1) to make the learning experience 
enjoyable, 2) to increase students’ interest in IS 
topics, and 3) to increase students’ understand¬ 
ing of IS topics. In sum, the purpose of the online 
forum discussion is to increase students’ learning 
in an enjoyable way. 

The remainder of the paper is organized as fol¬ 
lows. In section 2, we explain issues with the 
typical introductory IS course and discuss two 
pedagogical principles guiding the design of the 
online forum that we used. In section 3, we de¬ 
scribe how the forum was implemented in a high- 
enrollment class over three successive semesters. 
In section 4, we evaluate the effectiveness of us¬ 
ing the online forum and, in section 5, we discuss 
implications of online forum use. 
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DESIGN ONLINE FORUM DISCUSSION 

Issues with Introductory IS Courses 

While the content covered in an introductory 
IS course varies from university to university, 
the primary goal of the course is more or less 
the same: students need to obtain a basic under¬ 
standing of the technological, managerial, and 
societal aspect of IS. In other words, students 
need to understand how IS works, how IS can be 
used to facilitate business processes and manage¬ 
ment decision making, and what societal issues 
arise from use of an IS. To accomplish this goal, 
a variety of topics are typically covered: different 
types of information systems used by organiza¬ 
tions, database design and management, comput¬ 
er networking, wireless technology, the system 
development life cycle, and IS project budget¬ 
ing and management. The underlying problem 
of covering so many topics in a single course is 
that instructors are not able to offer in-depth 
discussion for any of the topics. Moreover, stu¬ 
dents who have different background knowledge 
or different interests in the various IS topics may 
prefer to delve deeper into some topics while sim¬ 
ply ignoring others. It is unlikely that in-class lec¬ 
tures and discussions, even when supplemented 
with first-rate text-books, will be able to meet all 
needs of all students. 

Another issue with the introductory IS class is 
that it is often too large, sometimes enrolling 
more than 100 students in a section. In such a 
large class, meaningful student engagement is 
difficult to accomplish. Typically, a handful of 
students will regularly participate in class discus¬ 
sions, but many students are simply unprepared, 
passive, or silent (Felder, 1997; Bhagyavati et ah, 
2005). To address both content and size issues, 
we provided our students with an online discus¬ 
sion forum where they were encouraged to ex¬ 
plore topics by posting links to recent IS-related 
articles found on the Web and by discussing the 
articles posted. 

Two Pedagogical Principles Guiding Our 
Design of Online Forum 

Two pedagogical principles guided our use of an 
online forum for the introductory IS course. The 
first principle was to engage students in active 
learning. As pointed out by Bakke and his col¬ 
leagues (Bakke, et. al, 2007) 


There is consensus among pedagogi¬ 
cal researchers that active learning 
techniques have a positive impact 
on the quality of students’ learning 
(Astin, 1984; Association of Ameri¬ 
can Colleges, 1986; Miller, 1988; 
Bonwell and Eison, 1991; House, 

2002; Kvam, 2002; McClanahan 
and MCClanahan, 2002; Udovic 
et ah, 2002). Active learning theory 
suggests that students become an 
integral part of learning process by 
studying ideas, solving problems, and 
applying what they learn. 

Since many students are not bold enough to 
speak up in a large section class, discussion often 
resembles a private conversation between the in¬ 
structor and a few of the most outgoing students 
rather than a broader dialog. Some students may 
be afraid of being embarrassed if they do not an¬ 
swer questions correctly or if they ask a “stupid” 
question. As a result, students often become pas¬ 
sive listeners rather than active participants in 
the class. Given the difficulty in providing a class¬ 
room environment that supports active learning, 
we chose to create an opportunity outside of the 
classroom that would encourages students to ac¬ 
tively seek information about course-related top¬ 
ics of interest. 

The second principle guiding our forum is to 
allow students to control some aspects of the 
learning process. In addition to choosing topics 
of interests for the online discussion, they can 
also choose the time and place to participate 
in the discussion. Researchers have shown that 
people are more motivated to do or complete a 
task when they have choices regarding the task 
(Bakke, et al, 2007, Malone, 1980; Lepper and 
Malone, 1987; Liao and Tai, 2006) or have an 
opportunity to regulate their learning process 
(Csikszentmihalyi, 2000; Lin and Hsieh, 2001; 
Zimbardo, 2005). 

USING ONLINE FORUM DISCUSSION 
TO FACILITATING LEARNING 

Gill (2006) listed several ways to use an online fo¬ 
rum to facilitate students’ learning: A forum can 
be used to support assignments so that students 
can share ideas. A forum can engage students in 
case discussion or debate. A forum can also be 
used as a tool for workflow control, archival stor¬ 
age, and monitoring, so that multiple classes over 
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several semesters can accomplish a large project. 
In our case, an online forum was primarily used 
for topical discussion. 

The potential for forum discussions to enhance 
student participation and learner outcomes has 
previously been established (Anderson and El- 
loumi, 2004, Corich et al., 2004). However, 
providing students with pertinent feedback 
and assigning a graded component for student 
contributions remains problematic - especially 
for high-enrollment classes (> 100 students) in 
which forum participation accounts for a signifi¬ 
cant portion of the overall grade in the course. As 
class size increases and instructional resources re¬ 
main fixed, the quality of feedback that students 
receive (if they receive any at all) inevitably de¬ 
teriorates to the point of being superficial (Rust, 
2001). To overcome these problems, the existing 
ratings feature in an open source forum prod¬ 
uct (JForum - www.jforum.net) was modified 
to serve as a peer assessment mechanism suited 
for our purposes. Although student peer assess¬ 
ment may at times be biased or more lenient than 
instructor feedback, it can potentially provide 
timelier and more voluminous feedback than a 
single instructor can muster - advantages that 
outweigh a certain degree of irregular quality 
(Nilson, 2002-2003). In addition, students’ ac¬ 
cumulated peer ratings were used as one factor 
in determining forum and course participation 
scores. 

We used the online forum for a large IS introduc¬ 
tory class in the fall 2006, spring 2007, fall 2007, 
and spring 2008 semesters. In each semester, two 
sections were offered, one in a lecture hall with 
a capacity of 180 seats and the other in a lecture 
room seating 140 students. The implementation 
and usage of the online forum varied slightly 
from semester to semester as we imposed addi¬ 
tional posting constraints in the second academic 
year and added a new features allowing students’ 
to view their “real-time” accumulated forum 
participation points online. Initially, there were 
two major posting areas on our forum: 1) “Lab”, 
where students were to post questions about lab 
sessions held over the course of the semester, and 
2) “IS Topics”, which were grouped into various 
subject areas such as hardware, software, IT and 
the environment, and IT and ethics. In practice, 
students seldom used the forum to discuss their 
lab assignments. As the lab discussion com¬ 
ponent was not a high priority for pedagogical 
purposes, we will therefore focus our analysis on 
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students’ forum use for dialogue on the various 
IS topics. 

In the first week of class in the Fall 2006 semes¬ 
ter, students were tasked with locating 6 current 
and unique online articles (no more than 6 weeks 
old) and then initiating discussion threads in 
the forum by posting the URL of the article, a 
short synopsis, and, most importantly, a personal 
reflection concerning implications for individu¬ 
als, organizations, or society at large. Students 
were also tasked with, over the course of the se¬ 
mester, reviewing and rating 18 of the articles (or 
threads) posted by their classmates. Peer ratings 
were indicated by a simple one star to five star rat¬ 
ing reflecting the degree to which the article and 
originator’s review was found to be interesting 
or helpful. Figure 1 presents an example discus¬ 
sion thread with associated reviews/ratings. The 
forum used for this study in one course section 
during the spring of 2008 can viewed online at 
http://cisr.cob.csuchico.edu/JForum2008S01/. 

For each discussion thread a student originated, 
the student earned 12 participation points. In 
addition, a student earned 4 points for each re¬ 
view/rating posted. If, on average, a discussion 
thread was peer rated at 3 stars, 2 additional 
points (14 pts. total) were rewarded to the origi¬ 
nator. A thread which averaged 4 stars received 
an additional 5 points (17 pts. total) and a 5-star 
thread received 9 bonus points (21 pts. total). 
Factoring students’ peer ratings into their over¬ 
all scores for forum participation was viewed as 
a means to encourage interesting and non-trivial 
contributions. Students were allowed to post bo¬ 
nus threads or reviews not to exceed 150 percent 
of the minimum requirements. In total, it was 
possible for a student to earn 300 points (30%) of 
their overall course grade through forum partici¬ 
pation. Previous research indicated that when 
used in an educational setting, forum participa¬ 
tion needs to account for a significant portion of 
a student’s overall grade or it will not be taken se¬ 
riously. Percentages between 30% and 50% have 
been suggested (Bhagyavati et al., 2005). 

Our early observation of students’ forum use was 
disappointing to us in that some students only 
input very brief comments about articles posted 
by others. Responses such as “great” or “sounds 
interesting” were not uncommon. To encourage 
students to input more detailed commentary, we 
imposed an additional requirement mid-semes¬ 
ter insisting that students provide postings of no 
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Figure 1 

Sample Forum Posting 
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fewer than 50 words when starting a new thread, 
and no fewer than 25 words when making a fol- 
low-on posting to an existing thread. Subsequent 
observation indicated that this requirement not 
only reduced the number of trivial postings but 
also reinforced the requirement that students 
support their opinions by citing additional facts 
or relating personal experiences. In subsequent 
semesters, we imposed these word length require¬ 
ments from the outset. 

For the 2006-07 classes, students needed to man¬ 
ually compute the points accumulated from their 
various postings or else wait until the instructor 
posted points after running a batch calculation 
process. In other words, even though students 
could calculate points for themselves, they did 
not have immediate access to a single score show¬ 
ing their total forum participation points. For 
the academic year 07-08 classes, the forum soft¬ 
ware was modified to calculate average ratings 
and point totals on an on-demand basis. This 
added feature was well received by students. 


The forum assignment was due to be completed 
anytime before final exams. Therefore, students 
could choose when to do the assignment - i.e. 
which articles to post and which threaded dis¬ 
cussions to respond to. This allowed students to 
control their learning experience in terms of time 
frame and content. It also encouraged students 
to engage in active learning: they had to explore 
the Web to find articles in which they were in¬ 
terested and make a judgment about whether or 
not the articles might be of interest to or generate 
higher ratings from their classmates. They would 
need to actively think about the contents of the 
articles since they had to summarize them and 
comments about them. Also, they had to read 
others’ posting and respond to those. The un¬ 
hurried, interactive atmosphere provided by the 
forum was expected to nurture a positive learn¬ 
ing environment and, ideally, increase students’ 
interest in the class and their understanding of 
IS topics. 
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Figure 2 

Postings by Week 
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EVALUATE EFFECTIVENESS OF 
ONLINE FORUM 

There can be both objective and subjective mea¬ 
sures of the effectiveness of using an online fo¬ 
rum. One objective measure is the number of 
participant logins. According to Gill (2006), if 
forum participation is voluntary, then usage or 
activity statistics become important indicators of 
success. In our study, forum participation was a 
requirement in all semesters. Moreover, Jforum, 
the particular forum software that we imple¬ 
mented, was not configured to track individual 
login events. Nonetheless, charting the number 
of forum postings created on a week-by-week 
basis does provide an indication of the level of 
student engagement. The two summary charts 
presented below depict the pattern of posting fre¬ 
quency over the 06-07 and 07-08 academic years. 

From the 06-07 posting chart, it is apparent 
that many students waited until very late in the 
semester to complete the forum assignment. In 
mid-semester, students were posting at a rate 
of only one entry every two weeks. In the final 
week of the semester, however, there were 3375 
postings - an average of 6 postings per student. 
We had hoped that our students would use the 
forum more consistently over the course of the 
semester, but, in fact, more than half of all post¬ 
ings were accomplished in the final two weeks of 
the semester. The observed frenzy of last-minute 
postings was disconcerting. In an attempt to 
counteract what we viewed as wholesale procras¬ 
tination, we programmed a change to the forum 
software to disallow posting of more than half 


of the overall requirements (3 articles and 9 re¬ 
views) in any one-week period. The effect of the 
additional constraint can be seen in the chart 
of AY 07-08 postings in which the number of 
postings in the final week of class is actually less 
than the prior weeks tally. Even so, forum activ¬ 
ity was still heavily weighted towards the last 3 
weeks of the semester. In order for students to 
have a better opportunity to engage in thought¬ 
ful discussions over the course of the semester, we 
believe that an even more finely grained posting 
schedule should be instituted and enforced by 
the software. 

In addition to objective measures of forum suc¬ 
cess, Gill (2006) suggested capturing subjective 
measures and points out that measures of a tech¬ 
nology’s impact on educational outcomes such 
as performance and satisfaction are important 
for situations in which participation is manda¬ 
tory. Gill (2006) further reports that a detailed 
analysis of individual student contributions is 
sometimes used to measure performance (e.g., 
Hazari, 2004; Gill, 2005). Characteristics such 
as length, breadth, depth and quality can be used 
to assess overall discussion performance (Benbu- 
nan-Fich, 2002). Although we did not perform 
detailed textual analysis of students’ postings, we 
did gather self-report data from students in the 
12 th week of the spring 2008 semester using an 
online survey instrument. The survey consisted 
primarily of 7-point Likert-style questions but 
also included one open-ended question for stu¬ 
dent comments. In most cases, we used multiple 
questions to assess student perceptions of forum 
characteristics. Although participation in the 
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survey was voluntary, students were encouraged 
to participate with an incentive of 25 class par¬ 
ticipation points, a 2.5% bonus. Students were 
required to have earned at least 50 points in the 
online forum before becoming eligible to com¬ 
plete the survey and the 50-point rule was en¬ 
forced by software. Across both sections of the 
course, 188 out of 287 students total completed 
the questionnaire - a participation rate of 65.5%. 

Our survey results indicated that students put a 
reasonable amount of effort into online forum 
discussion (Mean = 5.1) even though they did 
not report being particularly motivated to par¬ 
ticipate (Mean = 4.60). Generally, students rated 
online discussion quality quite positively along 
the dimensions of discussion breadth (Mean = 
5.27), discussion depth (Mean = 5.03), and over¬ 
all discussion quality (Mean = 5.28). 

Briggs, et al. (2003) suggested that subjective 
measures of user satisfaction are appropriate for 
cases involving group interaction. Our survey 
used two survey questions to estimate user sat¬ 
isfaction. Students reported being moderately 
satisfied with both the way the forum discus¬ 
sion was used (Mean = 5.19) and with how much 
they learned from the online discussion (Mean 
= 4.98). 

According to Gill (2006), the final category of 
effectiveness that should be considered is the de¬ 
gree to which a group discussion meets its design 
objectives. There were three objectives for our 
online forum: 1) Make the learning experience 
enjoyable, 2) increase students’ interest in IS top¬ 
ics, and 3) increase students’ understanding of IS. 
We used three questions to estimate attainment 
of objective of our first objective: 1) I enjoyed par¬ 
ticipating in the online forum discussion (Mean 
= 4.82), 2) the online forum discussion was an 
enjoyable way to learn about IS topics (Mean = 
5.09), and 3) I had fun reading the articles and 
comments posted on the online forum (Mean = 
5.07). Overall, our students agreed to a moderate 
degree that online forum discussion was enjoy¬ 
able. 

For objective #2, we used two questions: 1) In 
general, the online forum discussion increased 
my interest in IS topics (Mean = 4.76), and 2) 
as a result of participating in the online forum 
discussion, I become more interested in IS topics 
(Mean = 4.60). Overall, students reported only 
a slight increase in interest in IS topics resulting 
from their forum use. 


For objective #3 we used four questions to assess 
what students learned from the discussion: 1) 
Online forum discussion helps me to know the 
latest trends in computing technology, 2) Online 
forum discussion increased my understanding of 
technological aspect of IS (e.g., how a particular 
technology works), 3) Online forum discussion 
increased my understanding of managerial as¬ 
pect of IS (e.g., how can a technology facilitate 
business processes in organizations), and 4) On¬ 
line forum discussion increased my understand¬ 
ing of societal impact of IS (e.g., legal or ethical 
issues caused by IS). Student responses indicated 
that the online forum helped them learn the lat¬ 
est trends (Mean = 5.31) and societal impacts of 
computing technology (Mean = 5.10) more than 
technological (Mean = 4.82) and managerial as¬ 
pect of IS (Mean = 4.81). Students also moder¬ 
ately agreed with the statement “I obtained use¬ 
ful information that I would otherwise not able 
to obtain if not from online forum discussion” 
(Mean = 4.94). 

We conducted a correlation test between stu¬ 
dents’ overall evaluation of online forum qual¬ 
ity and the number of points earned to date 
(R 2 =0.15, p=0.022) and a correlation test be¬ 
tween points earned and learning outcome sat¬ 
isfaction (R 2 =0.18, p=0.027). The significance 
of the correlations indicate that students who 
had used the online forum discussion to a greater 
degree and had earned more points for doing so 
were more likely to report a favorable view of the 
online forum experience. 

To get a more in-depth understanding of stu¬ 
dents’ feedback, we added the following ques¬ 
tion at the end of the survey “Please write any 
additional comments”. Of the 88 students who 
provided additional comments, 3 wrote unre¬ 
lated comments, and only 7 expressed negative 
opinions regarding the forum assignment. The 
remaining 78 comments provided either neutral 
or positive overall evaluations. A summary of 
the positive remarks were that the online forum 
was easy to use, it was a fun and informative way 
to learn about IS, it helped students learn some¬ 
thing that they would not have explored on their 
own, and it was a good way to communicate and 
interact with other students in such a large class. 
The following are examples of comments made 
by our students. 

Forum helped all of the students engage 

in IT discussions 
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I found the forum very interesting. I 
really liked the responses I got back on 
some of my articles and it was nice to 
get other people’s opinions on certain 
topics. I think that the forum was user- 
friendly and informative. 

I enjoyed using the forum [for the class]. 

It enabled me to interact with my class¬ 
mates, learn interesting and new facts, 
and learn more about what is going on in 
the business world. 

It is a good idea because it forces you 
to learn something that you probably 
wouldn’t have if it weren’t for the forum. 

The Forum was a good way to gain infor¬ 
mation about new issues in the technol¬ 
ogy market. Instead of doing research 
yourself, it was a collaborative effort by 
the entire class and a whole cornucopia 
of different topics were brought up and 
you could find something interesting 
with relative little research. 

I thought the forums were very infor¬ 
mative. I really did learn a lot about 
current computer related issues that I 
would have normally never have learned. 

I think the forums were the best part of 
[the class]. 

Students also made suggestions regarding ways to 
improve the online forum discussion. A number 
of students observed that many of their class¬ 
mates waited until the last minute to participate 
in the online discussion and they suggested that 
there should be a means to spread the posting ac¬ 
tivity across the entire semester rather than have 
it cluster at the very end. Students also suggested 
that there should be improved quality control to 
ensure that posting are more relevant to the as¬ 
signed topics of discussion. Students further sug¬ 
gested that scoring their participation in the fo¬ 
rum should be more explicit and straightforward 
and that there should be fewer postings required 
of each student. Among the comments we re¬ 
ceived regarding students’ posting habits were 
the following: 

“I like the [forum] requirement. Maybe 
requiring a certain # of postings by the 
midterm might help spread out the post¬ 
ings a little more. 
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I think it would be more effective if the 
requirements were spread out more, for 
example having a post due once a month 
rather than four at the end. 

I did enjoy the forum, but thought there 
were almost too many postings required. 

I think that each student should have to 
post 2-3 articles, and maybe 8-10 replies. 

The point system was confusing. 

While the majority of the written comments 
were quite positive, we wondered why the stu¬ 
dents’ overall rating was lukewarm. There might 
be several explanations: First, those students most 
likely to take time to type in a specific comment 
might have been those who had the most posi¬ 
tive overall experience using the forum, whereas 
students who had negative overall perceptions 
of forum use may have chosen not to comment. 
Second, many students simply had little interest 
in the class or in the various IS topics. They took 
the class simply to fulfill a business school gradu¬ 
ation requirement and not out of any inherent 
interest. As a result, they may have cared little 
about the instructional methods or how much 
they actually learned. The following three com¬ 
ments provide examples: 

I understand the need for a forum and, 
for a class like this, it is an appropriate 
addition. But I did not enjoy being re¬ 
quired to post topics, especially when, 
even though I have respect for all com¬ 
puter topics, have very little interest in 
the topics. 

The forum is a cool feature. But it isn’t 
something that kept me interested in IT. 

This can be due to my already existing 
lack of interest in IT. 

I found the forum very challenging be¬ 
cause the information learned in this 
class does not interest me very much. I 
know that it is very useful, and you did 
a great job with the class! I think the fo¬ 
rum made me think about IS in a more 
critical way then I normally would have. 

In regards to the peer rating component of the 
forum assignment, only four students expressed 
negative comments explicitly about this feature. 
One student suggested that the grading should 
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be based entirely on the teachers evaluation and 
not on peer evaluations because the popular¬ 
ity factor forced students to choose articles that 
their college students peers would find inherently 
interesting. For example, postings regarding Ap¬ 
ple’s iPhones and other mobile devices were very 
popular as were postings regarding sustainability 
or green computing. 

In sum, while students’ overall evaluation of 
the online forum discussion was positive, there 
might not be too much room for instructors to 
increase student interests in IS topics where that 
interest is lacking before students enter the class¬ 
room. 

DISCUSSION 

A typical IS introductory course usually faces 
two challenges. One challenge is that there are 
too many topics to allow in-depth discussion of 
any particular topic. The other challenge is that 
class sizes are often too large to engage students 
in meaningful class participation. To address 
both challenges, we used a Web-based discussion 
forum complemented with an integrated peer¬ 
rating system for discussion of IS topics outside of 
the classroom. Based on statistics regarding stu¬ 
dents’ participation and students’ survey results, 
the forum was largely a success. First, the forum 
involved students in discussion to a greater ex¬ 
tent than classroom discussion did. For a typical 
high-enrollment class, classroom participation 
is limited to a small cadre of students, whereas 
our online forum discussion involved nearly all 
students in discussion of multiple topics. Sec¬ 
ond, according to our survey, students agreed to 
a moderate degree that the online forum discus¬ 
sion was enjoyable and that their interests in and 
learning about IS topics had been increased due 
to involvement in the forum discussion. The peer 
rating system was also well accepted by most stu¬ 
dents. In sum, the online forum did allow stu¬ 
dents to engage in a wider exploration of topics 
of interests, and many students encountered in¬ 
formation they otherwise might not have inves¬ 
tigated. 

Although the forum use appears to have been 
moderately successful, the following limitations 
should be borne in mind when evaluating our 
results. First, we did not record the number of 
logins for individual students. Doing so may 
have given us more insight into the degree of 
students’ engagement with the forum. We were, 


however, able to use timestamp data to track the 
rate at which postings were made over the course 
of the semester and we charted this on a week-by- 
week basis. Second, we did not conduct content 
analysis of students’ postings in order to assess 
the depth, breadth, and overall quality of forum 
discussion. Instead of having both objective and 
subjective measures, we used students’ self-report 
data to gauge discussion quality. 

Overall, students’ rating of the online forum 
discussion was lukewarm, and not as good as we 
had anticipated. We speculate that the reasons 
behind the lukewarm reception may include: 1) 
our students in general lack any inherent inter¬ 
ests in IS topics before enrolling in the class and, 
as a result, it was difficult to motivate them to 
engage more deeply in the learning process; 2) 
the content of the forum discussion was allowed 
to diverge into topics related to lecture content 
but not explicitly mentioned or covered in class 
lectures. Therefore, many students may not un¬ 
derstand all of the concepts needed for discus¬ 
sion of a particular IS-related topic brought up 
in the forum; and 3) many students postponed 
their engagement in the online discussions until 
the final weeks of class. This phenomenon re¬ 
sulted in a foreshortened span of time for many 
discussions and precluded students’ having the 
in-depth learning experience we had expected. 
As for issue #1, students’ lack of interest, in fu¬ 
ture studies we believe it will be informative to 
assess students’ interest in IS topics and IS classes 
both at the beginning of the semester and then 
again toward the end of the semester. This would 
allow for an estimate of the degree to which the 
online forum in particular and the IS course in 
general served to increase students’ interests in IS 
topics. In regards to issues #2 and #3, we antici¬ 
pate that requiring a weekly posting on a topic 
covered either in lecture or in the assigned read¬ 
ings might serve to supply additional motivation 
and to keep discussion focused on topics upon 
which all students should be able to converse. In 
addition, as instructors, we might also post some 
“seed” articles for a particular week for students 
to read and comment upon. By doing so, we have 
a ready mechanism to make available additional 
ad-hoc reading material that complements our 
basic course content. In addition, we have avenue 
by which to influence the course of “outside” dis¬ 
cussion. 

Despite the challenges discussed above, we be¬ 
lieve that online forum software can be used 
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(and also improved or customized) by other in¬ 
structors in order to increase student learning 
outcomes in an IS introductory course. We hope 
that sharing our experiences will give other in¬ 
structors (and developers) insights into the use 
of, and possible enhancements to the design of, 
online forum systems. 

REFERENCES 

Anderson, T and Elloumi, F (2004) “Theory and 
Practice of Online Learning,” Athabasca Uni¬ 
versity, Athabasca, Canada, retrieved March 
1, 2006. http://cde.athabascau.ca/online_ 
book/pdf/TPOL_book.pdf 

Association of American Colleges (1986) “A 
New Vitality in General Education,” Task 
Group on General Education. 

Astin, A. W. (1984) “Student Involvement: A 
Developmental Theory for Higher Educa¬ 
tion,” Journal of College Student Personnel, 
Vol. 22, pp. 297-308. 

Bakke, S., Faley, R. H., and Steinberg, G. (2007) 
“A Student-Centric Approach to Large Intro¬ 
ductory IS Survey Courses,” Journal of In¬ 
formation Systems Education, Vol.18, No.3, 
pp.321 - 328. 

Benbunan-Fich, R., Hiltz, S. R., and Turoff, M. 
(2002) “A Comparative Content Analysis of 
Face-to-Face vs. Asynchronous Group Deci¬ 
sion Making,” Decision Support Systems, Vol. 
34, pp.457-469 

Bhagyavati, Kurkovsky, S., and Whitehead, C.C. 
(2005) “Using Asynchronous Discussions to 
Enhance Student Participation in CS Cours¬ 
es,” ACM SIGCSE Bulletin, Vol.37, No.l, 
pp.111-115. 

Bonwell, C. C. and Eison, J. A. (1991) “Ac¬ 
tive Learning: Creating Excitement in the 
Classroom,” George Washington University, 
School of Education and Human Develop¬ 
ment, Washington, D C. 

Briggs, R. O., G.-J. d. Vreede and B. A. Reinig 
(2003) “A Theory and Measurement of Meet¬ 
ing Satisfaction,” Proceedings of the 36th 
Haiwaii International Conference on System 
Sciences, Big Island, Hawaii, IEEE. 

Corich, S., Kinshuk, and Hunt. L.M. (2004), 
“Assessing Discussion Forum Participation: 
In Search of Quality,” International Journal 


of Instructional Technology and Distance 
Learning, Vol.l, No. 12, pp. 3-12. 

Csikszentmihalyi, M. (2000) “Beyond Boredom 
and Anxiety,” 25 th Anniversary Edition. Jos- 
sey-Bass Publishings, San Francisco. 

Felder, R. M. (1997) “Beating the Numbers 
Game: Effective Teaching in Large Classes,” 
retrieved March 1, 2006. http://www.ncsu. 
edu/felder-public/Papers/Largeclasses.htm. 

Gill,T. G. (2005), “A Protocol for Online Case 
Discussions,” Decision Sciences Journal of In¬ 
novative Education, Vol. 3, No. 1, pp. 141-148. 

Gill, T.G. (2006) “Asynchronous Discussion 
Groups: A Use-based Taxonomy with Exam¬ 
ples,” Journal of Information Systems Educa¬ 
tion, Vol. 17, No. 4, pp. 373- 383. 

Hazari, S. (2004) “Strategy for Assessment of 
Online Course Discussions,” Journal of In¬ 
formation Systems Education, Vol. 15, No. 4, 
pp.349-355. 

House, J.D. (2002) “The Motivational Effects of 
Specific Teaching Activities and Computer 
Use for Science Learning: Findings from the 
National Mathematics and Science Study,” 
International Journal of Instructional Media. 
Vol. 29, No.4, pp. 423-439 

Kvam, P. H. (2002) “The Effect of Active Learn¬ 
ing Methods on Student Retention in Engin¬ 
eering Statistics,” The American Statistician, 
Vol. 54, No. 2, pp. 136-140. 

Lepper, M. and Malone, T. W., Eds. (1987) “In¬ 
trinsic Motivation and Instructional Effec¬ 
tiveness in Computer-based Education” Ap¬ 
titude, Learning, and Instruction. Erlbaum, 
Hillsdale, NJ. 

Liao, W. and Tai, W. (2006) “Organizational 
Justice, Motivation to learn, and training out¬ 
comes,” Social Behavior and Personality, Vol. 
34, No. 5, pp. 545-556 

Lin, B., and Hsieh, C. T. (2001) “Web-based 
Teaching and Learner Control: A Research 
Review”, Computer and Education, Vol. 37, 
pp. 377-386. 

McClanahan, E. B. and McClanahan, L. L. 
(2002) “Active Learning in Non-majors Biol¬ 
ogy Class,” College Teaching, Vol. 50, No. 3, 
pp. 92-94. 


Journal of Learning in Higher Education 


33 




Miller, G. E. (1988) “The Meaning of General Zimbardo, P. G. (2005) “Optimizing the Power 


Education: The Emergence of a Curriculum 
Paradigm,” Teachers College Press, Columbia 
University, New York. 

Udovic, D., Morris, D., Dickman, A., Postle- 
thwait, J. and Wetherwas, P. (2002) “Work¬ 
shop Biology: Demonstrating the Effective¬ 
ness of Active Learning in an Introductory 
Biology Course,” Bioscience, Vol. 52, No. 3, 

pp. 272 -281. 


and Magic of Teaching,” Journal of Social and 
Clinical Psychology, Vol. 24, No. 1, pp. 11-21. 


Appendix A 

Objective 1 (make learning experience more enjoyable): question 5, 6, and 7 
Objective 2 (increase students’ interest in IS topics): question 8 and 9. 

Objective 3 (increase students’ understanding of IS topics): questionl0, 11,12, 13. 
Usefulness of online forum: 14. 

Perceived motivation, 18 
Perceived effort: 16,17, 18, and 19 
Discussion quality: 20, 21, and 22. 

Peer rating feature: 23 
Peer rating quality: 24 and 25 
Satisfaction: 26 and 27 
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Survey of Online Forum Discussion 

1. How old are you?_ 

2. Sex: Male_Female_ 

3.1 am in Class_ 

4. How many years of full-time work experience do you have?_ 


3. I enjoyed participating in the online forum discussion. 

6. Online forum discussion was an enjoyable way to learn about IS topics. _ 

7.1 had fun reading the articles and comments posted on the online forum. 

8. In general, online forum discussion increased my interest in IS topics. _ 

9. As a result of participating online forum discussion, I become more interested 
in IS topics. 

10. Online forum discussion helps me to know the latest trend of computing 

technology. _ 

11. Online forum discussion increased my understanding of technological as- 

pect of IS (e.g., how a particular technology works) _ 

12. Online forum discussion increased my understanding of managerial aspect 
of IS (e.g., how can a technology facilitate business processes in organizations). 

13. Online forum discussion increased my understanding of societal impact of 

IS (e.g., legal or ethical issues caused by IS). _ 

14.1 obtained useful information that I will otherwise not able to obtain if not 
from online forum discussion. 

15.1 was motivated to participate in online forum discussion. 

16.1 made serious effort to participate online forum discussion. 

17. Before I wrote comments about a posted article, I usually read the original 
article in addition to the synopsis posted by my class mates. 

18. Before I wrote comments about a posted article, I usually just read the syn¬ 
opsis and not the original article. 

19. I always carefully evaluate the quality or usefulness of the article before I 
post it on online forum. 

20. The breadth of online forum discussion of IS topics is good. _ 

21. The depth of online forum discussion of IS topics is good. _ 

22. Overall, the quality of online forum discussion of IS topics is good. 

23.1 like the feature of peer rating of the online forum discussion. _ 

24. The ratings I have received for my postings in the forum seem reasonable to 
me. 

25. I am thoughtful and conscientious when I review and rate posting from 
other students. 

26.1 am satisfied with the way the forum discussion was used in this class. 

27.1 am satisfied with how much I learned from online forum discussion. 


28. Please write any additional comments in the following box. 


JLAJLAJL 

_3_4_5_6_7_ 

_3_AJ_AZ_ 

3 4 5 6 7 

3 4 5 6 7 

3 4 5 6 7 

3 4 5 6 7 

3 4 5 6 7 

3 4 5 6 7 

_3_4_5_6_7_ 

3 4 5 6 7 

3 4 5 6 7 

3 4 5 6 7 

_3_AJ_AZ_ 

JLAJ_AZ_ 

_3_4_5_6_7_ 

3 4 5 6 7 

3 4 5 6 7 

_3_4_5_6_7_ 

3 4 5 6 7 
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1 -Strongly Agree 









